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ABSTRACT 

I  he  probability  of  detection  (Pd)  of  targets  in  static  infrared  and  visually  cluttered  scenes  is  computed  using  the 
Fuzzy  Logic  Approach  (FLA).  The  FLA  is  presented  as  a  robust  method  for  the  computation  and  prediction  of  the  Pd  or 
targets  in  cluttered  scenes.  The  Mamdani/Assilian,  and  Sugeno  Neurofuzzy-based  models  have  been  mvesttgated.  A  large  set 
of  infrared  (IR)  imagery  and  a  limited  set  of  visual  imagery  has  been  used  to  model  the  relationships  between  several  input 
parameters;  the  contrast,  camouflage  condition,  range,  aspect,  width,  and  experimental  Pd.  The  fuzzy  and  neuro-fuzzy 
models  gave  predicted  Pd  values  tha,  had  0.98  correlation  to  the  experimental  Pd’s.  The  results  obtained  indicate  the 
robustness  of  the  fuzzy-based  modeling  techniques  and  the  applicability  of  the  FLA  to  those  types  of  problems  having  to  do 
with  the  modeling  of  human-in-the-loop  target  detection  in  any  spectral  regime. 


1.  IN  TRODUCTION 


More  than  three  decades  ago  Prof.  L.  A.  Zad.h  proposed  the  concept  of  tezy  logic  |  ,j  Following  Msmdani  anJ 
Assilian's  seminal  work  in  applying  ihe  funny  log.c  to  the  control  or  a  steam  engine  in  1974  [2],  the  FLA  has  been  finding  a 
rapidly  growing  number  of  applies, ions  ihroughout  Indus, ry  and  science.  These  applies, ions  include,  itansporiafion  (subways, 
helicopters,  elevators,  traffic  control,  and  a,r  conirol  for  highway  lunnels).  auloorobilcs  Icngincs.  brakes.  Iransrmssion.  and 
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cruise  control  systems  ),  washing  machines,  vacuum  cleaners,  rice  cookers.  VCRs,  air  conditioners,  microwave  ovens,  video 
cameras,  and  other  industries  including  steel,  chemical,  aerospace,  medical  diagnosis  systems,  information  technology,  and 
data  analysis,  ...etc.  [3, 4.  5,  6,  7). 

A  strong  point  ol  the  FLA  is  that  it  permits  the  encoding  of  expert  knowledge  directly  and  easily  using  rules  with 
linguistic  labels.  A  week  point  is  that  it  usually  takes  some  time  to  design  and  tune  the  membership  functions  which 
quantitatively  define  these  linguistic  parameters  of  interest.  It  has  been  found  that  artificial  neural  network  learning 
techniques  can  automate  this  process  and  substantially  reduce  development  time  while  improving  performance.  To  enable  a 
system  to  deal  with  cognitive  uncertainties  in  a  manner  more  like  humans,  researchers  have  incorporated  the  concept  of  fuzzy 
logic  into  these  systems  using  a  neural  network  approach.  The  integration  of  these  two  techniques  is  the  Ncuro-Fuzzy 
Approach  (NFA)  [S).  The  NFA  has  potential  to  capture  the  benefits  of  both  the  fuzzy  and  the  neural  network  methods  in  a 
single  model.  Ncuro-Fuzzy  modeling  basics  and  techniques  will  not  be  addressed  in  this  paper,  but  will  be  discussed  in  a 
later  paper.  A  new  approach  to  the  computation  of  the  probability  of  target  detection  in  infrared  and  visual  scenes  containing 
clutter  is  presented  here  At  present,  target  acquisition  models,  based  on  the  theory  of  signal  detection  theory,  arc  not  mature 
enough  to  robustly  model  the  human  detection  of  targets  in  cluttered  scenes.  This  is  because  our  understanding  of  the  visual 
world  is  a  result  ol  the  perception,  not  merely  detection,  of  the  spatio-temporal,  spectra-photomctric  stimuli  that  is  transmitted 
onto  the  photoreceptors  on  the  retina  [8].  The  computational  processes  involved  with  perceptual  vision  can  be  considered  as 
the  process  of  linking  generalized  ideas  or  concepts  to  retinal,  early  vision  data  [9|.  These  ideas  or  concepts  may  be 
represented  in  software  using  various  clutter  or  edge  metrics  [10]  as  well  as  luminance  attributes  of  a  military  vehicle  or 
automobile.  Fijom  a  system  theoretic  point  of  view,  human  perceptual  vision  involves  the  mapping  of  early  vision  data  onto 
one  or  more  concepts,  and  then  interring  a  meaning  of  the  stimuli  based  on  prior  experience  and  knowledge.  The  approaches 
ot  fuzzy  and  ncuro-fuzzy  systems  provide  a  robust  alternative  to  complex  semi-empirical  models  for  predicting  observer 
responses  to  visual  and  IR  cluttered  scenes.  The  fuzzy-based  approaches  have  been  used  to  calculate  the  probability  of 
detection  (Pd)  of  vehicles  in  different  infrared  and  visual  scenes. 


1  Further  author  information  - 

T.I.M.  (correspondence):  Email:  rneilzlet@cc.tacom.army.mil; Telephone:  (810)574-5405;  Fax  (810)574-6145 


2.  PROBABILI I  \  OF  DETECTION  (Pd)  FOR  GROUND  TARGETS 

Dunng  the  past  three  decades  there  has  been  an  increasing  interest  in  the  development  and  use  of  computational 
models  to  compute  the  static  and  dynamic  probability  of  detection  of  a  target  in  visual  and  infrared  images  that  include  a 
known  or  unknown  amount  of  Cutter  HO.  1 1,12.13).  Clutter  and  the  idea  of  the  probability  of  detection  play  a  very  important 
role  in  the  area  of  machine  vision  and  human  in-thc-loop  target  acquisition. 

The  problem  of  modeling  electro -optical  ,EO)  sysrems  for  ,hc  purpose  of  ground  vehicle  countermeasures 
development  and  system  performance  modeling  is  an  old  problem  1 14|.  There  are  many  algorithms  used  for  the  computation 
of  the  probability  of  detection  of  a  targel  in  clutter  currently  used  in  the  Itterature  of  image  processing  and  target  acquisition 
modeling.  Most  are  some  variation  of  the  Ft.  Bclvoir  .  Virginia  Nigh,  Vision  Lab  (NVL)  model.  During  the  pas,  trvemy 
years  there  has  been  an  increasing  interest  in  the  modeling  community  adapt  the  methods  and  results  of  recent  rcseateh  in 
the  area  of  neurophysiology  and  human  vision  research  ,„,o  mul, .resolution  target  acquisition  modeling  1 15, 16.231,  Presently, 
target-acquisition-model  strategics  can  be  divided  into  three  broad  classes: 


' '  '  C  °mplllCr  modC,S  ‘hat  Use  ,he  “classkar  “PProach  to  modeling  target  acquisition  performance.  These  models 
usnall,  assume  a  targe,  of  uniform  conlras,  Co  ,he  bnckground.  The  temperarnre  or  luminance  of  ,hc  tnrgel  is  computed  by 
avenging  over  nil  the  potcls  on  the  target.  In  these  classical  models,  the  targe,  si*  and  comma,  are  rake,  tut  ,he  most 
emporium  signature  paramc.crs  for  predicting  iarge,  deice, ability  b,  observers.  This  type  of  model  has  both  IR  and  visual 

versions.  Signal  Detection  Theory  fSDT)  is  used  with  this  approach  1171  to,  rhe  compmarion  of  Receive  Opera,,, 
Characteristic  (ROC)  curves. 

(2)  Computational  models  that  use  the  multi-channel  and  multi-resolution  idea  carried  over  from  human  vision 
research  [24-28]  together  with  signal  detection  theory  concepts  from  classical  psychophysics.  The  input  scene  is  considered 


as  forming  modulated  signals  that 


arc  transduced  by  the  eye  and  visual  cortex  to  form  a  mental  image  from  which  an  observer 


sees  or  delects  a  rargei.  This  is  the  so-called  "b<„,„ms-up  "  approach  based  on  firs, -principles  of  human  vision  and 

psychophysics.  These  kinds  of  computer  modcis  arc  visual  models  to  star,,  bu,  can  he  applied  to  IR  scenes  since  the  eyc  is 
assumed  to  look  at  a  displayed  image  on  a  monitor  1 18.  24-28). 

(3)  Computer  models  that  use  neural  networks  and/or  fuzzy  logic  for  predicting  detectability  based  on  the  input  of  a 


data  set  of  target  “feature  vectors." 


from  radar  and  acoustics. 


This  type  of  computer  model  can  be  used  for  both  IR  and  visual  images,  as  well  as  images 


3.  FUZZY  MODELS 

l  uzzy  modeling  is  an  approach  based  on  fuzzy  logic  with  fuzzy  predicates,  using  a  description  language.  Zadeh's 
principle  of  incompatibility  [29]  states  that  “...as  the  complexity  of  a  system  increases,  our  ability  to  make  precise  and  yet 
significant  statements  about  its  behavior  diminishes,  until  a  threshold  is  reached  beyond  which  precision  and  significance  (or 
relevance)  become  mutually  exclusive  characteristics." 

l-u/./y  models  basically  (all  into  two  categories  [30].  The  first  category,  linguistic  models,  is  based  on  the  collection 
01  II  Thcn  rules  witil  vasuc  Predicates,  and  use  a  fuzzy  reasoning  such  as  Mamdani's  and  Assilian's  model  [31],  In  these 
models,  sets  of  rules  operating  with  linguistic  values  or  input/output  variables,  appear  as  an  analogy  to  the  system  of  equations 
used  lor  description  of  linear  and  non-linear  systems.  For  a  Two-Inpul-Single-Output  (TISO)  system,  the  model  has  the  form: 

II  x  is  A  and  y  is  B  Then  z  is  C  ^  j\ 

Where  A.  B.  and  C  are  fuzzy  sets  of  the  universes  of  discourse  X.  Y.  and  Z.  respectively. 

Sugeno  s  model  [32 1  is  the  other  category  of  fuzzy  models  and  they  are  characterized  with  functional  type 
conclusions.  The  Sugeno  model  for  a  TISO  system: 

If  x  is  A  and  y  is  B  .  then  z  =  f(x,  y)  (2) 

where  A,  and  B  are  fuzzy  sets  of  the  universes  of  discourse  X  and  Y  respectively, 
and  x  and  y  are  values  of  the  input  variables. 

A  first-order  Sugeno  has  the  form  [33J. 

If  x  is  A  ahd  y  is  B  Then  z  =  px+qy+r  .  (  3) 

Fuzzy  models  can  be  developed  using  the  direct  approach,  where  the  model  is  made  directly  from  an  expert's 
knowledge  and  expressed  in  the  form  of  logical  rules.  The  system  identification  method  is  another  approach  [30],  which  is 
based  on  the  input-output  data  using  classical  system  identification  techniques  and  neural  networks.  This  approach  may  be 
accomplished  using  the  structure  and  parameter  identifications  [32].  The  structure  identification  using  Clustering  techniques 
including  Fuzzy  t  -Means  (FCM)  [34|,  the  Mountain  Clustering  [35],  and  the  Subtractive  Clustering  [36)  methods;  determines 
input  and  output  variables,  relationships  between  the  variables,  number  of  rules,  and  the  partitioning  or  the  input  and  output 
variables  into  fuzzy  sets.  The  parameter  identification  estimates  the  MF's  of  the  fuzzy  set.  More  details  on  the  Fuzzy  Logic- 
Approach  as  applied  to  target  acquisition  modeling  may  be  found  in  (19). 


4.  IMPLEMENTATION 


THc  deVC,0pment  01  lhc  software  Packagc  that  models  the  relationship  between  the  various  factors  that  affect  the 
determination  of  the  probability  of  detection  is  currently  undergoing  development.  Some  of  these  modeling  factors  are 
introduced  here  to  demonstrate  the  capability  of  the  fuzzy  and  neuro-fuzzy  approaches  in  predicting  Pd  with  a  0.9  correlation 
to  experimental  values  as  shown  m  Fig.  2.  Tables  I  and  2  show  some  of  the  signal,  clutter  metrics,  and  the  and  the  signal-to- 
no.se  ratio  (SNR),  and  relat.vc  SNR's  [  1 1]  of  the  target  in  the  foreground  of  the  IR  image  set  of  which  Fig.  I  is  an  example. 
I  he  Probability  of  detection  (Pd)  value  can  be  determined  with  the  FLA  using  input  parameters  Tor  the  images  shown  in  Fig. 
I  and  Fig.'s  3  through  8.  the  one  output  parameter  ,s  Pd.  Fig.  I  ,s  a  sample  of  the  infrared  terrain  board  images,  from  Or.  B. 
O  Kane  at  NVESD,  used  in  this  study.  The  terrain  and  vehicles  arc  scaled  models  of  natural  terrain  and  military  ground 
vehicles.  NVESD  is  the  U.S,  Army  center  For  research  on  infrared  systems.  Fig.'s  3  through  8  are  from  Dr.  Lex  Toe.  of  TNO 
in  the  Netherlands  [37  J.  TNO  is  a  non-profit  research  agency  the  performs  human  factors  research.  The  figures  show  a  jeep- 
like  vehicle  in  a  field  at  a  distance  of  around  500  meters.  These  figures  were  shown  to  observers  in  a  perception  lab  and  the 
observers  responses  were  aggregated  to  obtain  experimental  Pd's.  For  the  infrared  images,  there  was  enough  data  to  take 
every  other  data  point  and  then  compare  the  model  predicted  Pd  to  the  other  set  of  data  points  not  used  in  the  training.  In  the 
case  of  the  visual  images,  we  expanded  the  dataset  of  6  initial  images  and  metrics  to  21  sets  of  metrics  by  taking  averages 
below  and  above  each  individual  metric.  This  expanded  dataset  was  then  used  as  input  to  the  MATLAB  Fuzzy  Logic 
Artificial  Neural  Fuzzy  Inference  System  (ANF1S).  The  ANFIS  (22)  uses  neural  network  algorithms  to  assist  in  the  formation 

of  membership  functions  and  rules  thereby  saving  the  user  a  large  amount  of  time.  ANFIS  was  used  to  generate  the  FIS  that 
computed  the  Pd  for  each  of  the  images. 


l  ig.  I  Night  Vision  Laboratory  Terrain  Board  thermal  image 
courtesy  of  Dr.  Barbara.  O’ Kane 


ExpjxJ 

— Fuzzy  pd 


1  ig.  _  Lixperimental  vs.  Fuzzy  Logic  Approach  predicted  Pd  for  infrared  terrain  board  imaiges 
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Table  1  Signals  for  IK  images,  difference  between  target  and  background. 


DTI 

DT2 

DT3 

DT4 

DT5 

scenar 

pix avg dt 

rss dt deg 

doyle dt 

awaa dt 

moulton dt 

Rel DT 

Norm DT 

1607 

1.042 

1.053 

1.047 

1.042 

4.333 

1.488 

6.452 

1609 

-0.142 

0.431 

0.221 

0.142 

0.32 

-4.63 

0.335 

1619 

0.715 

0.891 

0.745 

0.715 

3.237 

-0.5 

4.461 

1645 

0.718 

0.786 

0.718 

0.718 

3.287 

-0.85 

4.115 

1657 

0.395 

0.702 

0.445 

0.395 

2.106 

-2.58 

2.385 

2127 

1.244 

1.263 

1.26 

1.244 

5.119 

3.155 

8.12 

2151 

0.625 

0.704 

0.636 

0.625 

2.631 

-1.68 

3.288 

2155 

0.864 

0.886 

0.865 

0.864 

3.334 

-0.06 

4.902 

2159 

-0.619 

1.049 

0.829 

0.619 

2.631 

1.295 

6.26 

2165 

0.233 

0.405 

0.265 

0.233 

1.152 

-4.47 

0.495 

2171 

1.191 

1.261 

1.191 

1.191 

5.279 

3.016 

7.981 

2665 

-0.828 

1.222 

1.072 

0.828 

3.833 

3.64 

8.605 

2669 

-1.488 

1.701 

1.605 

1.488 

6.318 

9.635 

14.6 

2677 

-0.098 

0.473 

0.222 

0.098 

0.406 

-4.6 

0.363 

2691 

0.771 

0.796 

0.776 

0.771 

3.328 

-0.62 

4.347 

2697 

-0.368 

0.834 

0.416 

0.368 

2.512 

-1.2 

3.76 

3169 

-0.292 

1.285 

0.987 

0.292 

1.789 

0.937 

5.901 

3179 

-0.379 

0.547 

0.393 

0.379 

1.366 

-2.61 

2.351 

3185 

-1.175 

1.819 

1.614 

1.175 

5.594 

8.665 

13.63 

3189 

-0.683 

0.767 

0.706 

0.683 

2.772 

0.44 

5.405 

3205 

0.417 

0.509 

0.433 

0.417 

1.66 

-3.37 

1.595 

3209 

0.161 

0.405 

0.214 

0.161 

0.499 

-4.96 

0 

3211 

1.273 

1.338 

1.282 

1.273 

5.709 

3.734 

8.698 

3213 

0.096 

0.608 

0.424 

0.096 

0.58 

-3.86 

1.103 

Table  2.  Noise  terms,  various  clutter  metrics  and  relative  SNR  of  the  IR  images 


der 

poe 

schmeider 

mean_bkg 

std^bkg 

REL CLUT 

Norm RCL 

REL  SNR 

Pd-Exp 

188 

0.001 

19.001 

3.797 

0.251 

0.60806 

3.85806 

2.44647 

0.897 

174 

0.001 

19.204 

4.353 

0.238 

-1  5894 

1.66057 

2.91252 

0.385 

166 

0.003 

22.004 

2.921 

0.323 

5.79679 

9.04679 

-0.0869 

1 

163 

0 

15.581 

3.62 

0.307 

0.98251 

4.23251 

-0.8646 

0.846 

159 

0.004 

19.428 

2.955 

0.375 

5.35337 

8.60337 

-0.4819 

0.923 

278 

0.002 

15.743 

3.45 

0.421 

1 .35949 

4.60949 

2.32096 

0.538 

288 

0.003 

17.071 

3.456 

0.204 

0.21512 

3.46512 

-7.7934 

0.59 

322 

0.004 

15.204 

3.462 
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figures  3  through  8  below  show  the  visual  band  TNO  images  used  for  the  FLA  approach. 


The  images  are  of  a  truck  and  jeep 


at  a  distance  of  approximately  500  meter 


s  at  various  aspect  angles.  In  some  of  the  pictures  the  vehicle  is  partially  concealed 


behind  a  hill  or  'berm'  or  tree. 


In  Table  3  are  listed  some  of  metrics  for  the  Fig.'s  3  through  8.  The  column  labeled  Ll/Lb  is 


the  target  luminance  divided  by  the  background  luminance  which  is  the  effective  SNR  for  these  images. 


Visual  Images 

Courtesy  of  Dr.  Lex  Toet  of  TNO 
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fable  3  Visual  Image  metrics  and  signal  to  noise  ratio  iLt/I.hi 
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Below  in  Fig.  9  is  shown  the  Fuzzy  Identification  System  (FIS)  that  we  constructed  in  MATLAB  with  the  input  parameters 
mentioned  above  and  the  Pd  as  output.  The  input  parameters  are  shown  on  the  far  left  of  the  figure  to  be;  the  width  of  the 
vehicle,  the  target  luminance,  the  background  luminance,  'nmrs‘.  which  is  a  texture  metric,  the  distance  to  the  vehicle  from 


the  observer,  the  aspect  angle  of  the  vehicle  relative  to  the  observer  and 


weather  or  not  the  vehicle  has  camouflage  on  it. 


The 


center  block  in  Fig.  9  shows  the  type  of  FIS  being  constructed,  and  the  box  on  the  far  right  shows  the  output  variable  name. 
Fig.  10  is  the  tiring  array  for  the  various  membership  functions  and  can  actually  be  manipulated  real  time  by  the  user  to  get  a 
visual  feeling  lor  how  changing  the  values  of  the  parameters  effects  the  averaged  Pd  output.  Each  column  in  Fig.  10 
represents  a  parameter,  such  as  target  range,  and  each  row  represents  a  group  of  membership  functions.  The  column  on  the 
far  right  shows  the  Pd  averaged  over  each  membership  function. 
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Fig.  9.  Fuzzy  Logic  Identification  System  for  computing  visual  Pd’s 


Fig.  10  Firing  diagrams  for  the  FTS 


5.  RESULTS 

I  he  percentage  Pd's  predicted  using  the  Neuro-Fuzzy  approach  are  shown  below  in  Fig.  1 1  The  correlation  of  the 
experimental  Pd  to  the  FLA  predicted  Pd  was  0.99.  What  is  interesting  is  that  a  0.9  correlation  was  achieved  using  an  input 
dataset  of  only  5  images  with  seven  metrics  for  each  image.  This  result  is  indicative  of  the  power  of  using  the  FLA  to  model 
highly  complex  data,  for  which  there  would  be  many  interrelated  equations  if  one  tried  to  model  the  detection  problem  in  th<j 
standard  algorithm  based  method,  in  one  relatively  simple  model. 


F*g-  1 1  Bar  chart  showing  the  comparison  of  experimental  Pd  to  FLA  predicted  Pd  for  visual  images 


6.  CONCLUSIONS 


in  conclusion,  the  FLA  yields  very  satisfactorily  results,  and  requires  a  fraction  of  the  effort  that  goes  into  traditional 
algorithm  based  techniques  of  modeling  target  acquisition  probabilities.  Furthermore,  fuzzy-based  solutions  can  be  created  in 
weeks  to  months  in  comparison  to  the  years  that  may  be  needed  to  create  a  traditional  solution.  We  expect  that  the  fuzzy 
modeling  approach  could  be  used  in  the  statistical  decision  theory  modules  of  target  acquisition  models  for  any  spectral 
regime. 

We  have  developed  a  prototype  software  using  the  fuzzy  logic  approach.  Two  fuzzy  models  have  been  used:  namely 
the  Mamdani  and  Sugeno  models.  For  the  visual  image  scenarios,  using  only  5  input-output  pairs,  we  were  able  to  predict  the 
Pd  values  with  a  correlation  of  0.99  as  shown  in  Fig.  II.  A  correlation  of  0.98  was  achieved  for  the  image  set  of  IR  images. 
As  shown  in  Tables  1.  2.  and  3.  the  signal-lo-noise  level  becomes  close  to  zero  in  many  cases  and  the  FLA  still  produced  a 
model  with  high  correlation. 

This  application  of  the  FLA  involved  pictures,  metrics,  and  experimental  Pd’s  taken  in  the  infrared  and  visual  band. 
Future  work  will  involve  the  application  of  the  FLA  to  predict  the  Pd's  of  moving  targets  in  visual  and  infrared  cluttered 
scenes  and  the  inclusion  in  the  Fuzzy  Inference  System  of  multifrequency  and  color  plane  information  The  membership 
functions  can  be  designed  using  experimental  Pd's  collected  in  the  TARDEC  Visual  Perception  Laboratory  (VPL)  |23|.  As 
mentioned  in  [23],  the  TARDEC  VPL  is  being  used  in  a  collaborative  R&D  project  with  an  auto  company  on  vehicle 
conspicuity  and  for  various  U.S.  Army  work  related  to  visibility.  The  lab  was  recently  established  and  is  a  2500  sq.  ft.  facility 
that  permits  embedded  simulation.  The  lab  is  equipped  with  state-of-the-art  eyetracking  and  head  tracking  devices  as  well  as 
high  resolution  color  projectors.  The  use  of  the  lab  tp  obtain  experimental  Pd's  is  important  to  ihe  process  of  building  the 
fuzzy  logic  models  and  calibrating  other  computational  vision  models  that  emulate  human  perception.  The  more  the 
experimental  data  available  the  more  robust  will  he  the  membership  functions.  Having  a  laboratory  to  perform  perception  test 
frees  one  from  the  problems  with  field  tests  such  as.  inclement  weather  and  the  transportation  of  equipment.  Recently  the  lab 
has  been  used  in  several  instances  to  collect  observer  data  concerning  visibility  of  various  systems. 

The  FI -A  can  also  be  applied  to  detection  problems  in  different  energy  areas,  for  instance  acoustic  energy 
propagation  and  material  (i.e.,  metal  or  semiconductor  characterization).  The  FLA  applied  to  acoustic  detection  problems 
will  be  the  topic  of  a  future  paper. 
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